Linguistics is the scientific study of language. One aspect of linguistics is the application of computer science to human natural languages such as English. Due to the greatly increased speed of processors and capacity of memory, computer applications of linguistics are on the rise. For example, computer-enabled analysis of language discourse facilitates numerous applications such as automated agents that can answer questions from users. The use of “chatbots” and agents to answer questions, facilitate discussion, manage dialogues, and provide social promotion is increasingly popular. To address this need, a broad range of technologies including compositional semantics has been developed. Such technologies can support automated agents in the case of simple, short queries and replies.
But such solutions are unable to leverage rich discourse related information to answer questions, perform dialog management, provide recommendations or implement “chatbot” systems, because existing solutions are unable to match an answer with a question due to insufficient rhetorical analysis. More specifically, statistical based solutions are unable to separate the task of determining a topic from a sentence and addressing rhetorical agreement between a sentence and an answer. Statistical-based solutions either do not consider rhetorical structure of a question and a response at all, or attempt to address topic and rhetorical agreement simultaneously and fail to properly address rhetorical agreement. Without sufficient rhetorical analysis, questions, which can have arbitrary rhetorical structure, cannot be matched with appropriate answers, which may also have arbitrary rhetorical structure.
Moreover, existing solutions are unable to analyze noisy text such as text that includes grammatical errors or variants such as colloquialisms, slang, or shorthand used in online discussions. Hence, new solutions are that can accurately represent rhetorical agreement between a question and an answer in the case that the question or answer is noisy.